The Effects of Communication Style and Message Function in Triggering 
Responses and Critical Discussion in Computer-Supported Collaborative 

Argumentation 



Allan C. Jeong 
Florida State University 



Abstract 

This study examined how differences in communication style affect how likely particular types of messages 
(e.g. arguments, evidence, critiques, explanations ) were able to elicit critical responses during four online 
debates. Event sequence analysis was used to compare the response probabilities for each type of message 
across messages that used expository versus epistolary styles of communication observed in the four 
asynchronous threaded discussions. The results suggests that when a message is posted to challenge an 
opposing viewpoint, that message is significantly less likely to elicit a return response from the opposition when 
the message acknowledges individuals by name or presents a direct reference to an individual’ s preceding 
statements. A more detailed and exploratory analysis of the interactions revealed that this style of 
communication might have contributed to a decrease in the frequency of evidence and subsequent discussion of 
supporting explanations needed to defend the challenged viewpoints and arguments. 

Introduction 

Computer- mediated communication (CMC) provides opportunities for learner-to-learner interaction 
and enables learning communities to engage in more reflective critical thinking. Argumentation is one of the 
fundamental collaborative inquiry -based learning strategies for increasing critical thinking skills in online 
settings. These skills involve the processes of building arguments to support a position, and considering and 
weighing evidence and counter-evidence in developing supporting arguments. Innovative approaches to 
facilitating student participation in collaborative argumentation have been developed in computer-supported 
collaborative learning (CSCL) and computer-supported collaborative argumentation (CSCA). One approach to 
scaffolding and facilitating collaborative argumentation is to structure the discussions such that constraints are 
placed on when and what types of functional moves and messages can be posted to discussions (Cho & 

Jonassen, 2002; Duffy et ah, 1998; Jonassen & Remidez, 2001; McAlister, 2001). The other approach 
incorporates intelligent systems and pedagogical agents that use formalized models of argumentation to actively 
diagnose students' performance and suggest immediate courses of action (Eleuterio, Barthes & Bartolozzi, 

2002; Jacques, Oliveira, & Vicari, 2002a; Jacques et. Al, 2002b; Karacapilidis & Papadias, 2001). 

Despite these technical advances to support the functions students perform during online discussions 
and regardless of what technology is used to facilitate discussion, a growing body of research now suggests that 
student participation in online discussions can be influenced by the communication styles of its participants. 
When and how often a student responds to a message may depend not just on what is said in the message (in 
terms of function of the message), but also by how the message was delivered in terms of communication style. 
Significant differences in communication styles have been found between males and females. For example, men 
tend to assert opinions strongly as facts, place more value on presenting information using an expository style, 
are more likely to use crude language, violate online rules of conduct, engage in more adversarial exchanges, 
and terminate exchanges when there are disagreements (Fahy, 2002; Herring, 1993; Savicki et al., 1996). In 
contrast, females are more likely to qualify and justify their assertions, use expressions that convey more 
epistolary roles, make apologies, and in general, manifest a more consensus-making orientation and epistolary 
style. Furthermore, females are more upset by violations of politeness and are more likely to challenge 
participants that violate rules of conduct and (Smith, McLaughlin & Osborne, 1997). Although each gender may 
have a tendency towards one communication style, groups have also been found to modify their communication 
styles in the direction of the majority gender (Baym, 1996; Herring, 1996). 
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However, the impact of communication styles on when and how often messages are able to elicit 
responses that generate more critical and more substantive discussions has yet to be examined and empirically 
investigated. The lack of empirical research can be attributed to the absence of appropriate theories, methods 
and tools capable of operationalizing and producing precise measurements, descriptions and assessments of 
group interactions and processes in online discussions (Fahy, 2001 & 2002; Garrison, 2000). The main 
problems lie in the coding and analysis of computer conference messages (Rourke, Anderson, Garrison, Archer, 
2001) and the sequential nature of messages and responses (Gunawardena, Lowe & Anderson, 1997). The 
difficulty in coding message content is in establishing the unit of analysis because messages often contain 
multiple ideas that serve multiple functions. As a result, the contents of a message must be classified into 
multiple codes, making it nearly impossible to map message-response sequences in terms of pre -defined 
message categories (Levin, Kim & Riel, 1990; Newman, Johnson, Cochrane & Webb, 1996; Gunawardena, 
Lowe & Anderson, 1997). These methodological problems have prevented researchers from conducting a closer 
and more detailed examination of the relationships between messages and responses - particularly the 
functional, temporal, social and semantic relationships between exchanged messages and responses (Jeong, 
2004a). 

One solution to this problem is to require students to pre -classify their contributions to discussions 
using a pre -determined set of message/response categories. This constrains each message to serve only one 
function at a time, and establishes a clear unit of analysis. This approach has been used in a number of 
computer-supported collaborative argumentation (CSCA) systems (Cho & Jonassen, 2002; Duffy et ah, 1998; 
Jonassen & Remidez, 2001; McAlister, 2001). The ACT system (Sloffer, Dueber & Duffy, 1999), for example, 
is a threaded discussion board that is designed to scaffold online debates by requiring students to pre -classify 
each posting to one of six response categories - proposal, counter-proposal, supporting reasons, detracting 
reasons, supporting evidence and detracting evidence. McAlister (2003) proposed a synchronous chat tool to 
support collaborative argumentation by requiring students to pre -classify messages to inform, question, 
challenge, reason, support or maintain chat discussions. Within each of these response categories, students are 
able to choose a specific sentence opener (e.g. "A counter-argument is...") to channel students' thoughts by the 
process of completing the sentence in a way that fits with the opener. 

Purpose 

Using the methods described above, the purpose of this study was to determine the precise 
probabilities in which messages with a given function (e.g. arguments, supporting evidence, critiques and 
explanations) and communication style (epistolary vs. expository) are able to successfully elicit responses that 
contribute to discussions that critically examine claims and alternative viewpoints. The research questions that 
were specifically addressed in this study were the following: 

1. Level of Interaction Between Opposing Viewpoints. When conducting discussions to critically 
examine alternative viewpoints, how does the use of epistolary styles of communication affect how 
likely students will respond back to students who challenge their claims with counter-arguments and 
supporting explanations? 

2. Impact on Level of Critical Discussion. What are the implications of using versus not using these 
specific styles of communication on the level and depth of discussion across all presented arguments 
and claims? 



Method 

Participants 

The participants (n = 17) were graduate students from a major university in the Southeast region of the 
U.S., consisting of 1 1 female and 6 male students, and ranging from 20 to 40 years in age. The students were 
enrolled in a 16 -week online graduate introductory course to distance learning required to complete a Master's 
program in instructional systems with a major in distance learning. The gender of each student was determined 
at the beginning of the online course through personal introductions and posted biographies. 

Debate procedures 

Students were required to participate in a series of four highly structured debates using threaded 
discussion forums in Blackboard, a web-based course management system. The discussions were highly 
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structured given that: a) student participation in the debates and other scheduled discussions throughout the 
course contributed to 20% of the course grade; b) for each debate, students were required to post at least four 
messages to receive full credit for each debate; c) prior to each debate, students were randomly assigned to one 
of two teams (but balanced by gender) to either support or oppose a given position; and d) students were 
required to vote on the team that presented the strongest arguments after each debate. 

The purpose of each debate was to critically examine design issues, concepts and principle in distance 
learning covered during the week of the debate. For example, students debated over the following claims: “The 
Dick & Carey ISD model is an effective model for designing the instructional materials for this course”, “The 
role of the instructor should change when teaching at a distance”, “Type of media does not make any significant 
contributions to student learning”, and “Given the data and needs assessment, the fictitious country of NED 
should not develop a distance learning system”. Students were instructed to support and refute presented claims 
and viewpoints with arguments, evidence, explanations, and critiques. 

Online debate messages and message labels 

Students were presented a list of four response categories during the debates designed to scaffold 
argumentation. The response categories were based loosely on Toulmin’s (1958) model of argumentation. The 
response categories and their definitions were presented to students prior to participating in the debates. Each 
student was required to classify each posted message by response category by inserting the corresponding label 
into the subject headings of every message. Students were required to limit the content of their messages to 
address one and only one response category at a time. The experimenter occasionally checked the message 
labels to determine if students had appropriately labeled their messages according to the described procedures. 
No participation points were awarded for a given debate if a student failed to properly label one or more posted 
messages. However, students were able to return to a message at any time to correct for errors in their labels. 



Symbol 


Description of symbol 


+ 


Identifies a messaae costed bv a student assianed to the team sucoortina the 
given claim/statement 


- 


Identifies a messaae costed bv a student assianed to the team oooosinathe 
given claim/statement 


ARG# 


Identifies a messaae that oresents one and onlv one araument or reason for 
using or not using chats (instead of threaded discussion forums). Number each 
posted argument by counting the number of arguments already presented by 
your team. Sub-arguments need not be numbered. ARG = "argument". 


EXPL 


Identifies a reply/message that provides additional support, explanation, 
clarification, elaboration of an argument or challenge. 


BUT 


Identifies a reply/message that questions or challenges the merits, logic, 
relevancy, validity, accuracy or plausibility of a presented argument (ARG) or 
challenge (BUT). 


EVID 


Identifies a reply/message that provides proof or evidence to establish the 
validity of an argument or challenge. 



Figure 1 - Example instructions on how to label messages during the online debates 

The purpose of labeling and constraining each message to a specific function was to make the inter- 
relationships between students’ contributions explicit and to enable students to visualize the structure of their 
arguments (Jeong & Juong, 2004b). The second reason for using these procedures is that they enabled the 
experimenter to clearly establish each message as a unit of analysis. With each message assigned to a specific 
response category, the types of message-response sequences observed in the debates could be clearly identified 
and counted to determine their relative frequencies. Previous studies on group interaction in CMC did not 
succeed in measuring message-response sequences (Gunawardena, Lowe, & Anderson, 1997; Newman, 
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Johnson, Cochrane, & Webb, 1996) because students’ messages and responses often addressed multiple 
functions or response categories at the same time. As a result, mapping the relationships between messages and 
responses was a difficult if not impossible task. In this study, however, the use of student-labeled messages was 
found to be an adequate solution to resolving some of the previous problems by establishing a clear unit of 
analysis. Although these procedures appear at first sight to be artificial and perhaps intrusive, this method has 
been implemented in a number of computer-supported collaborative argumentation (CSCA) systems for 
scaffolding argumentation and problem solving (Carr & Anderson, 2001; Cho & Jonassen, 2002; McAlister, 
2003; Sloffer, Dueber, & Duffy, 1999; Veerman, Andriessen, & Kanselaar, 1999). 

In addition, students were also instructed to identify each mess^e by team membership by adding an 
for opposing or a “+” for supporting team at the end of each label (e.g. +ARG, -ARG). These tags allowed the 
students to easily locate the exchanges between the opposing and supporting teams within the discussion 
threads (e.g. +ARG->-BUT) and respond to the exchanges to advance or defend their team’s position. An 
example discussion thread from a debate is illustrated in Figure 2. 

Figure 2. Example of online debate with labeled messages 



r 


E SUPPORT statement because... 


Student names 


Sat Oct 2, 2004 11:18 am 


r 


B +ARG#1 MedialsButAMereVehicle 


Student names 


Mon Oct 4, 2004 8:47 pm 


r 


E-EVID MedialsButAMereVeh... 


Student names 


Tue Oct 5, 2004 7:09 pm 


r 


E +But RelativitvTheorv... 


Student names 


Tue Oct 5, 2004 9:43 pm 


r 


-But RelativitvThe. . . 


Student names 


Sat Oct 9, 2004 10:12 am 


r 


-BUT Whataboutemotions? 


Student names 


Tue Oct 5, 2004 9:53 pm 


r 


+EVID DistEdEffectiveAsF2F 


Student names 


Tue Oct 5, 2004 10:40 pm 


r 


-BUTMediaamerevehicle 


Student names 


Wed Oct 6, 2004 8:19 pm 


r 


E +EVID MooreConcurs 


Student names 


Wed Oct 6, 2004 10:07 pm 


r 


+EXPLMediaSelectionCo... 


Student names 


Sun Oct 10, 2004 12:35 am 


r 


B -BUT WellChosenEffect. . . 


Student names 


Sun Oct 10, 2004 4:31 pm 


r 


+But SunnortinaRes... 


Student names 


Sun Oct 10, 2004 5:37 pm 


r 


B -BUTMediaismorethenamere... 


Student names 


Fri Oct 8, 2004 5:30 pm 


r 


+BUT SuoDortinaEviden... 


Student names 


Sat Oct 9, 2004 8:51 am 


r 


-BUT LearninaNotSimolvAP... 


Student names 


Mon Oct 1 1 , 2004 9:54 am 


r 


B +ARG2 Standards for teachina 


Student names 


Wed Oct 6, 2004 1:48 pm 


r 


+But Clarification? 


Student names 


Sun Oct 10, 2004 5:39 pm 


r 


B +ARG3 MediaUnrelatedtoLearn... 


Student names 


Wed Oct 6, 2004 3:12 pm 


r 


B -BUTMediaUnrelatedtoLear... 


Student names 


Wed Oct 6, 2004 8:26 pm 


r 


B +BUT MediaSelection 


Student names 


Thu Oct 7, 2004 9:20 am 


r 


-BUT MediaSelection 


Student names 


Sun Oct 10,2004 11:21 am 


r 


+EVID MethodNotMedia 


Student names 


Wed Oct 6, 2004 1 1 :04 pm 


r 


B-BUT MediaUnrelatedtoLea... 


Student names 


Sat Oct 9, 2004 10:59 am 



Data analysis 

Inter-rater reliability. To determine the extent in which the messages were labeled correctly by students, 
two of the four debates were randomly selected and coded by the experimenter to determine inter-rater 
reliability (work in progress). 

Data preparation prior to analysis. To prepare the data for analysis, computer software was written by the 
experimenter to download, tabulate and compile the student -labeled messages from the Blackboard discussion 
forums into Microsoft Excel. The experimenter reviewed all the messages to identify and tag any messages that 
contained expressions that reflected more conversational and epistolary exchanges. The expressions that were 
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found to exhibit the epistolary style were direct references to previous statements (e.g. “I agree when you say 
that...), and references to individual’s names (e.g. “Hi Bob”). All messages that contained these expressions 
were tagged with an “s” (e.g. BUTs). As a result, a total of eight message categories (ARG, ARGs, BUT, BUTs, 
EVID, EVIDs, EXPL, EXPLs) were used to code the messages. 

Event sequence analysis. Event sequence analysis (Bakeman & Gottman, 1997) was used to identify and 
quantify the observed interaction patterns between messages that exhibited different communication styles. 
Sometimes referred to as lag analysis or Markovian chain analysis, the purpose of event sequence analysis is to 
determine: a) the probability in which a given event is able to elicit one or more subsequent events; and b) the 
probabilities in which specific types of events are likely to follow a given event. Event sequence analysis has 
effectively used in communications research to study, for example, communication patterns in the conversations 
and interactions between married couples (Bakeman & Gottman, 1997 pp. 184-193; Gottman, 1979), children at 
play (Bakeman & Brownlee, 1982), mother and infant at play (Stern, 1974), and humans and computer- 
interfaces (Olson, Herbsleb, & Rueter, 1994). 

A computer program (Jeong, 2003), the Discussion Analysis Tool (DAT), was developed by the 
experimenter to perform the event sequence analysis. DAT tallied the frequency of each message-response 
interaction. The response frequencies for each target message were then converted into relative frequencies, or 
transitional probabilities (Bakeman & Gottman; 1997). The transitional probabilities measured how likely a 
particular type of response was posted in reply to a particular type of message relative to all the other types of 
responses elicited by the given message. 

Transitional state diagrams. The observed transitional probabilities between messages with different 
communication styles and function were converted into transitional state diagrams using DAT. In the diagrams, 
each response category was represented in a node. Directional arrows were drawn from one node to another 
node to represent the relative frequency of each observed message-response interaction. The density of the 
directional arrows connecting the nodes illustrates the strengths of the transitional probabilities between the 
nodes. This graphical representation of the interactions provided a means to readily identify prevalent patterns 
of interaction produced by messages that exhibited expository styles and epistolary style. Most of all, the 
diagrams provided a visual approach to identifying differences in message-response sequences between the 
message types. 



Results 

Summary statistics and overall level of interaction. Figure 3 shows the frequency matrix that summarizes 
the number of observed messages and responses across categories. The four debates produced a total of 323 
messages. Of the 323 messages, 239 were posted in reply to a previous message and 166 of the messages did 
not elicit any replies. The matrix also shows how many messages of a given category were observed, and how 
many responses were elicited by the message within a given category. For example, the debates generated a 
total of 84 arguments (ARG), 65 messages (BUT) that challenged its merits with out using an epistolary style 
versus 20 challenging messages (BUTs) with epistolary style. In Figure 4 is the transitional probability matrix, 
which shows the relative frequency of each type of response elicited by each message category. For example, 
50% of responses to ARG were BUT (65 of the 129 replies) versus 16% BUTs (20 of the 129 replies). This 
matrix also shows the overall response rates for each message category. For example, 81% of the arguments 
elicited a response based on the finding that only 16 of the 84 did not receive any responses at all. The overall 
response rate across all message categories was 67%. 
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Figure 3 - Matrix with message and response frequencies 
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Figure 4 - Matrix with transitional probabilities between messages and responses 
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Level of interaction between opposing viewpoints. By analyzing only the interactions produced by 
exchanges between participants on opposing teams (e.g. +ARG^ -BUT, -BUT->+BUT, -BUT->+EVID, etc.), 
the results showed that messages presented with personal acknowledgments and direct references to an 
individual’s previous statements (epistolary style) received significantly fewer return responses (or rebuttals) 
from the opposition (X = .39, STD = .60, n =67) compared to messages that were presented without using this 
style (X = .61, STD = .83, n = 256), t(321) = 2.08, p = .038, with the effect size of -.31. 

Impact on level of critical discussion. To explore the potential impact of using the epistolary style of 
communication on the subsequent direction and level of critical discussion, the transitional probabilities from 
the upper-left and lower-right quadrants in the probability matrix (Figure 4) were examined separately to 
compare the types of discussion that would or could be produced if all messages in a discussion were presented 
with versus without the epistolary style. The transitional probabilities produced by messages without the 
epistolary style (upper right quadrant) are depicted in the left diagram in Figure 5. The diagram on the right 
depicts the interactions produced only by messages presented with the epistolary style. 



399 



Figure 5 - Possible patterns of interaction when discussions consist only of expository messages versus only 
epistolary messages. 





Expository -> Expository Epistolary -> Epistolary 

message-response exchanges message-response exchanges 

A comparison of the two diagrams reveals possible differences in outcomes following interactions 
produced with epistolary style versus without epistolary style of communication. When the epistolary style was 
not used, challenges (BUT) directed at an argument (ARG->BUT) were more likely to elicit responses with 
evidence in defense of the challenged argument. In other words, the sequence of events produced without the 
epistolary style (ARG-^BUT^EVID) was more likely to occur (in 15% of replies to BUT) when the epistolary 
style was not used to challenge an argument. When the epistolary style was used to present a challenge 
(ARG^BUTs), this interaction was less likely to lead to the production of evidence (in only 8% of replies to 
BUTs). The left diagram in Figure 5 also suggests that once evidence was elicited by a challenge (without the 
epistolary style), students were more likely to sustain and develop the discussion with explanations (ARG 
BUT EVID -> EXPL -> EXPL). These differences suggest that the use of expository style produced 
discussions that exhibited more reflection and critical examination of arguments during the online debates. 
However, these observations must be considered with caution because these observations are based on low 
frequencies of messages presented with the epistolary style. 

Discussion 

The findings of this study show that the epistolary style of communication - namely the direct reference to 
individuals by name and direct reference to previous statements - inhibited the critical discussion and 
examination of arguments, and that critical discussion is more likely to occur when students adopt and 
implement a more expository rather than epistolary style of discourse during computer-supported collaborative 
argumentation. This finding may appear to contradict the prediction that a more epistolary or conversational 
style of communication encourages individual participants (particularly female participants) to reciprocate 
responses and extend a conversation by helping to build rapport between participants (Fahy, 2002). However, 
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the findings in this study showed that the epistolary style inhibited rather than supported student responses to 
challenges. One possible explanation for this unexpected finding is that the confrontational nature of 
collaborative argumentation may have been intensified or even personalized when students made direct 
reference to individual names and previous statements. Engaging in discussion without this epistolary style 
would seem to have helped the students maintain focus on the ideas and not the personalities involved in the 
discussions, thereby reducing the sense of contentiousness inherent in the debate activity. 

The findings from the exploratory analysis also suggest that the use of the epistolary style could have a 
potentially negative effect on the level of reflection, critical analysis and discussion. The analysis of the state 
diagrams revealed patterns to suggest that the epistolary style can reduce the frequency of evidence and 
explanations presented in defense of arguments following challenges to the arguments. To thoroughly test the 
effects of discourse styles on students’ interaction patterns and the outcomes produced by the interactions, 
future studies will (a) need to collect a larger corpus of data to gather sufficient number of messages with 
contrasting styles, or (b) conduct a controlled experiment in which students in multiple groups are instructed to 
implement different styles of discourse as they participate in online collaborative activities such as debates, 
problem-solving, and case-studies. 

In conclusion, this study was successful in making a first attempt at determining the effects of discourse 
styles on student performance in CMC when no previous studies have succeeded or attempted to determine its 
actual impact. The findings in this study serve to demonstrate the efficacy of using event sequence analysis, 
combined with the use of response constraints to label messages by function, to precisely measure and study the 
effects of epistolary versus expository styles in computer-mediated communication. Ultimately, this study will 
also serve as a model for investigating the effects of other styles and protocols commonly observed in online 
communication, which include the use of emoticons, qualifiers, humor, rhetorical questions, and the explicit 
versus discrete expression of disagreements. Measuring the precise effects of these communication styles and 
protocols will enable researchers and instructors to develop more refined strategies and computational models 
for supporting and optimizing student performance in computer-supported collaborative learning. 
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